Method for the Assessment of Possible Trajectories

ABSTRACT

A method for assessing possible trajectories of road users in a traffic environment includes capturing the traffic environment with static and dynamic features, identifying at least one traffic user, determining at least one possible trajectory for at least one road user in the traffic environment, and assessing the at least one determined possible trajectory for the at least one road user with an adapted/trained recommendation service and the captured traffic environment.

This application claims priority under 35 U.S.C. § 119 to patentapplication no. DE 10 2019 209 736.7, filed on Jul. 3, 2019 in Germany,the disclosure of which is incorporated herein by reference in itsentirety.

The disclosure relates to a method for assessing possible trajectoriesof road users in a traffic situation

BACKGROUND

An assessment of the trajectory options of other road users is essentialfor reliable trajectory planning of an at least partially automatedvehicle in real traffic situations. For example, if another vehicleenters a common lane, a vehicle that is at least partially automated mayhave to reduce speed. If a pedestrian wants to cross the road in frontof the automated vehicle, it may have to come to a complete standstill.

Known classification models have a fixed number of classes. Theseclasses each constitute a specific trajectory for a road user for a typeof crossing and are highly specialized in this one case. For example, ageneral problem is simplified into a fixed set of options, whereinheuristics are used or machine learning techniques are used to calculatea specific trajectory.

SUMMARY

One problem with the solutions indicated in the prior art is that notall options of possible traffic situations are covered, which can leadto overly cautious or, on the other hand, overly dangerous planning. Thelarge number of possible real-world scenarios cannot be described withvariants of types of environment defined in advance.

With too many different classes of trajectories, trajectory planningsystems based on class classification quickly reach functional limitsdue to the large number of possibilities. This applies in particularwhere the classes are difficult to distinguish and may not be properlyannotated.

This is because agents influence each other, including the actions ofthe automated ego-vehicle. This leads to a combinatorial explosion ofpossible alternatives and the generation of a set of rules of allpossible solutions is beyond practical possibilities.

Also to create heuristics, i.e. a set of learned rules which are appliedin certain situations and can be location-dependent orscenario-dependent, can quickly be unachievable with the number ofdifferent locations and scenarios.

When using state-of-the-art machine learning approaches to estimateagent trajectories, i.e. trajectories of road users, these trajectoriesare generated, for example, by reinforcement learning, reinforcementlearning, recurring neural networks, or imitation learning.

One problem with this approach is that the trained model must receiveand understand all the necessary inputs and must then be able to outputone or a plurality of different trajectories or intentions that complywith traffic law. However, this large number of calculated trajectoriesis then difficult to process further without assessing the individualtrajectories. The results thus produced by the machine learning systemare not sufficiently reliable and are difficult to validate.

When simplifying the approach of estimation of trajectories bydetermining a limited set of possible classes raises the problem thatnot all options are covered, which may result in planning that is toocautious or dangerous. This is because the planning system must obtainas many options as possible from the other agents and must be able toweight them in accordance with their probability and impact in order tobe able to plan comfortable and safe trajectories.

According to the disclosure, a method, an apparatus and a computerprogram product and a computer-readable storage medium are specified forthe assessment of possible trajectories of road users of a trafficenvironment according to the features of the independent claims, whichat least partially have the aforementioned effects. Advantageousembodiments are the subject matter of the dependent claims and thefollowing description.

The disclosure is based on the finding that an assessment of aprobability that trajectories of a particular traffic environment, whichare possible from a traffic technology point of view and according totraffic law, are driven by a road user, and thus constitute observedtrajectories, may be carried out by means of a recommendation service ina particularly suitable manner.

In that regard, such an assessment must be taken as a recommendation ofa trajectory to the traffic environment which is using therecommendation service, so to speak, and the road user is part of thetraffic environment.

The basic assumption of a recommendation service is that similar usersbehave similarly. The assessment of the users is implicitly derived fromthe use of the trajectories and a metric.

A user assessment implicitly results regarding the future behavior ofthe users by comparing the possible trajectories with the actualobserved trajectory. This will be repeated multiple millions of timesover an extended period for a large number of traffic environments. Oneadvantage of this method is that multiple almost equally probableassessments are possible without it leading to ambiguities, for examplein that a method of another type predicts an averaging of twotrajectories which is unfavorable and can lead to an impossibletrajectory.

According to one aspect of the method for assessing possibletrajectories of road users of a traffic environment, in one step thetraffic environment is captured with static and dynamic features.

In a further step of the method, at least one road user is identified.In particular, this identification can be carried out by using thecaptured dynamic features of the traffic situation.

In particular, a road user can be identified by using the captureddynamic features of the traffic situation.

In a further step of the method, at least one possible trajectory isdetermined for at least one road user in the traffic situation.

In a further step of the method, the determined at least one possibletrajectory for the at least one road user is assessed by means of anadapted recommendation service and the captured traffic environment.

The possible trajectories of road users are the trajectories that therespective road user may travel in compliance with the traffic rules. Inparticular, however, it may also be the trajectories that are possiblefor the respective road user to travel. The environment will bespecified relative to the respective road user. For example, theenvironment may be specified centered around the road user.

A recommendation service is a method that aims to make a prediction andto quantify how great is a user's interest in an object, in order torecommend to the user exactly the objects from the set of all existingobjects in which he is most likely to be interested.

Collaborative recommendation services, also known as collaborativefilters, recommend the objects in which users with similar assessmentbehaviors, i.e. similar users, have the greatest interest. In order todo this, there is no need for further knowledge about the object itself.Such a collaborative recommendation service is therefore able toaccurately recommend complex objects without the need for an“understanding” of the object itself.

A recommendation service assesses objects based on given informationabout the current usage based on previously considered user-objectcombinations based on metrics.

Collaborative filtering systems have many forms, but many common systemscan be reduced to two steps:

-   -   Searching for users that share the same assessment patterns with        the active user, that is the user for whom the prediction is        being determined.    -   Using the assessments of the users found in the search to        calculate a prediction for the active user.

This falls under the category of user-based collaborative filtering. Aspecific application is the user-defined K-nearest neighbor algorithm.

One advantage of this method for assessing possible trajectories oftransport components is that it is not subject to any restriction inrelation to the traffic environment. No rules or heuristics need to beexplicitly developed. Such an assessment of possible trajectories can bedone quickly, almost in real time.

Static and dynamic features of the traffic environment can relate todetails of static and dynamic objects, but these features, for exampleas a whole, can also relate to the entire object itself.

The identification of a road user can be carried out by means of thecaptured dynamic features of the traffic situation.

The captured traffic environment, with static and dynamic features,means the capture of the features at a specific time at which theinformation is available to carry out this method.

An adapted recommendation service is a recommendation service that hasbeen set up and tested in relation to its structure, i.e. the type ornature of the recommendation service, and in relation to its specificdesign, for example by training a neural network or by establishing aK-nearest neighbor method by means of training data or training vectorsand similar, to perform the specific task. Example structures or theadaptation of such a recommendation service will be described in moredetail.

According to one aspect, it is proposed that the traffic environment hasthe following dynamic and static features or objects of a trafficsituation: speeds, speed differences, mutual distances between roadusers, distances of the road users from the environment, plan views ofroad users and traffic situations, directions of motion or orientationsof road users relative to the environment.

Furthermore, the traffic environment comprises the following dynamic andstatic features or objects of a traffic situation: traffic rules, inparticular locally relevant traffic rules, stop lines, stop lines,traffic lines, lane markings.

In addition, the traffic environment has the following dynamic andstatic features or objects of a traffic situation: traffic light status,geographical map data, lanes, structures of intersections, junctions andtraffic routes in general; and additionally summarized descriptions of atraffic environment such as: traffic situations in general, trafficflow, traffic jam, clear road.

Road users refers not only to vehicles in the general sense, but also toall road users, especially pedestrians, with their position,orientation, and speed.

According to one aspect, it is proposed that at least one feature of thetraffic environment is captured by means of a sensor.

These can be static or dynamic features, which can also be used tocapture temporary features or temporal processes.

According to one aspect, it is proposed that the traffic environment becaptured by determining at least one dynamic feature at two or moreconsecutive times.

If there is successive information for the description of the trafficenvironment in each individual assessment step, the accuracy of themethod can be significantly increased.

According to one aspect, it is proposed that features of the trafficenvironment should comprise at least one traffic rule and/or at leastone feature of at least one traffic light system and/or at least onestatic or dynamic road sign.

This additional information describes the traffic environment in moredetail and thus allows a better assessment of the possible trajectories.

According to one aspect, it is proposed that at least one feature of thetraffic environment is captured by means of an imaging sensor.

Such an imaging sensor allows at least parts of the traffic environmentto be identified and classified very precisely, which in turn can have apositive effect on the accuracy of the assessment.

According to one aspect, it is proposed that at least one feature of thetraffic environment is captured by means of a sensor of an ego-vehicle.Advantageously, this type of capture of the traffic environment isfocused on a particularly relevant part, in relation to the ego-vehicle.

According to one aspect, it is proposed that at least one feature of thetraffic environment is captured by means of a stationary sensor. Forexample, such a sensor, at busy or complex-structured intersections, maytransmit data to, for example, an ego-vehicle, which enable theego-vehicle to improve the assessment of possible trajectories, inparticular of other road users.

According to one aspect, it is proposed that at least one dynamicfeature of the traffic environment is captured by means of a sensor.Compared to other options for capturing dynamic features, such a sensorcan increase reliability. For example, a radar sensor can directlydetermine the speed of a road user. But dynamic features or objects canalso be derived from specific image processing of imaging sensors, forexample.

According to one aspect, it is proposed that at least some of the staticfeatures and/or the dynamic features of the traffic situation of atleast one road user will be transmitted wirelessly to another road userin the traffic situation.

This allows major traffic situations to be surveyed by the interactionof multiple road users, and, for example, a traffic environment ortraffic situations that are difficult to see to be made available tosome road users.

According to one aspect, it is proposed that the recommendation servicewould be adapted to assess possible trajectories using a large number ofcorresponding combinations of assessed observed trajectories, differenttraffic environments, and at least one possible trajectory.

This means that the recommendation service can be built up on the basisof observed trajectories linked to the appropriate traffic environmentsin order to adapt the recommendation system. Observed trajectories aretrajectories that have been travelled in the real world in a specifictraffic environment and can be represented and recorded in many ways.

According to one aspect, it is proposed that the recommendation serviceis a collaborative recommendation service and has a neural network withat least one convolution layer or has a recursive neural network or isbased on a K-nearest neighbor method.

According to one aspect, it is proposed that the neural network has afirst autoencoder for capturing the environment and has a secondautoencoder for capturing the possible trajectories.

By dividing the neural network in the manner described, the training canbe made easier.

An “autoencoder” is understood to be an artificial neural network thatallows certain patterns contained in the input data to be learned.Autoencoders are used to generate a compressed or noise-freerepresentation of the input data by suitably extracting the essentialfeatures, such as certain classes, from the general background.

The autoencoder uses three or more layers:

-   -   An input layer, such as a 2-dimensional image.    -   Multiple significantly smaller layers that form the encoding for        reducing the data.    -   An output layer, the dimension of which corresponds to the input        layer, i.e. each output parameter in the output layer has the        same significance as the corresponding parameter in the input        layer.

Alternatively, the traffic environment and the possible trajectories canalso be entered directly into a neural convolutional network (CNN:convolutional neural net) to obtain an assessment.

According to another aspect, it is proposed that the vectors of theK-nearest neighbor method are formed with a variety of correspondingcombinations of captured traffic environment and observed trajectory.

The K-nearest neighbor method provides a parameter-free andeasy-to-implement approach to assessing new objects based on previouslyexplicitly assessed and stored objects.

According to one aspect, it is proposed that at least one static featureof the traffic environment is captured by means of a geographical map.This is a simple method for capturing traffic-relevant static features.With highly accurate geographical maps, the method can be improvedfurther.

According to one aspect, it is proposed that the traffic environmentwill be captured by transforming spatial parts of the trafficenvironment into a two-dimensional reference system, corresponding to aplan view of the traffic environment.

A plan view or a top view is a graphically represented, two-dimensionalorthogonal projection of a spatial situation. In contrast to a bird'seye view, the plan view is a planar representation from which thedimensions of the object can be measured.

According to one aspect, it is proposed that the captured trafficenvironment includes a previous trajectory of at least one road user.

According to another aspect, it is proposed that at least one dynamicfeature of the traffic environment of at least one road user will becaptured in its previous course for a sufficiently long period of timeto enable a further course of the trajectory to be estimated therefrom.

According to one aspect, it is proposed that the captured trafficenvironment includes the speed of at least one road user.

According to one aspect, it is proposed that the captured trafficenvironment includes the direction of movement of at least one roaduser.

According to one aspect, it is proposed that at least one possibletrajectory is determined by an optimization of cost functions and/or bymeans of a search-based method and/or a machine-learning method.

By means of this variety of methods for determining the possibletrajectories, the method can be adapted to local conditions.

The determination of possible trajectories can be carried out byheuristics for lane-keeping, for speed control and according to thecourse of the road. For example, a possible trajectory is determined forall possible lanes, which are supplemented with other trajectories,which take into account different traffic light circuits.

Standard methods can generate possible trajectories based onoptimization of cost functions, can be search-based, or can use machinelearning. The calculation of a trajectory by optimization creates atrajectory iteratively or directly which has very small deviations fromthe best possible features, such as driving comfort, distance from roadusers and complying with traffic rules. Search-based methods, in simpleterms, seek to iteratively generate a best trajectory for their currentenvironment by searching for a connection from A to B. Machine learningmethods can learn to imitate already seen trajectories and generate newtrajectories based thereon.

Alternative trajectory calculation functions are, for example,optimization-based and look for a trajectory which minimizes a givencost function; map-based methods simply only follow the lanes, forexample; deep learning methods such as “imitation learning”,“reinforcement learning” generate a trajectory by means of machinelearning; physics-based methods determine trajectories based on thedynamics of the vehicle and are mainly used for short approximations.

In particular, a trajectory depicts a location of an object againsttime, can but also alternatively or additionally be represented as anindication of a speed against time and/or an acceleration against time.

When determining trajectories, all possibilities of change of direction,corresponding to the lanes and merging of lanes, are taken into account.

In particular, a possible trajectory for each road user can bedetermined by means of an accurate map of the respective environmentincluding intersections, junctions, the road layout, and/or the currenttraffic regulations.

According to one aspect, it is proposed that at least one possibletrajectory is determined taking into account driving comfort on thetrajectory and/or a distance from other road users and/or compliancewith traffic rules.

In particular, in the case of at least partly automated vehicles whichuse the described method to determine the ego-trajectory to be driven,for example, this serves to improve safety in the traffic situation ortraffic environment.

According to one aspect, it is proposed that at least one possibletrajectory is determined by means of a map representation of the trafficenvironment. In this case, all options for changing the direction oftravel, corresponding to the lanes and merging of the lanes, as well asintersections, and current traffic regulations are to be taken intoaccount.

According to one aspect, a method for the generation of a recommendationservice for the assessment of possible trajectories is proposed.

In one step, a large number of corresponding combinations of thecaptured traffic environment, observed trajectory and at least onepossible trajectory is determined for a large number of differenttraffic environments by repeating the following steps: In one step, thetraffic environment is captured with at least one static feature and atleast one dynamic feature.

In a further step, at least one road user in the traffic situation isidentified.

In a further step, an observed trajectory of the at least one road useris captured.

In a further step, at least one possible trajectory for the at least oneroad user in the traffic environment is determined.

Then, the recommendation service with the large number of correspondingcombinations of the captured traffic environment, the observedtrajectory and the at least one possible trajectory is adapted by usinga deviation of the observed trajectory from the at least one possibletrajectory.

Sensor data for capturing the traffic environment can be captured usingfused sensor data.

According to one aspect, it is proposed that for the assessment of anobserved trajectory-traffic environment combination in a capturedtraffic environment, a difference of a possible trajectory of atransport road user from an observed trajectory is calculated using ametric. Due to the result of applying the metric to the deviation of anobserved trajectory from a possible trajectory, the trajectory used canbe described quantitatively and stored for further calculation.

For the training data, all trajectories are explicitly assessed using ametric. The recommendation service will then seek to implicitly minimizethis metric, corresponding to a similarity metric, by means of theassessments.

For example, a system determines a series of possible trajectories basedon the current road map. All possible changes of direction based onexisting lanes and possible merging of lanes are calculated. Thereafter,a series of features of the traffic environment, for example relativedistances, speeds, number of road users, their relationships, trafficrules, stop lines, traffic routes, plan views of road users, etc., isused as input for a recommendation service for estimating theprobability of the possible trajectories.

Offline, i.e. without the influence of an active prediction system, thepossible trajectories and observed trajectories of all road users arecompared. The individual possible trajectories can be assessed by usingdifferences between lane position, speed deviations, etc. for multipletime steps.

These assessments are used to train a machine learning system to

-   -   a) estimate the assessment of an intention as a regression        problem,    -   b) compare the probability of two possible trajectories by        classifying the possible trajectory with a higher probability,        and    -   c) specify estimated probabilities for each possible trajectory.

This trained or adapted recommendation system, also referred to as arating estimator, is then used online to estimate probabilities of thedifferent intentions.

According to one aspect, it is proposed that the metric for calculatingthe difference of a possible trajectory of a road user from an observedtrajectory determines a distance from waypoints on the respectivetrajectories.

According to a further aspect, it is proposed that the metric is theabsolute difference from points on the respective trajectories at threedifferent times, wherein a possible trajectory is calculated and addedto the observed trajectory after a time when the traffic environment iscaptured. For example, the three different times can be 1, 2 or 3seconds after determining the possible trajectory and can result frombalancing cost and benefits.

According to another aspect, it is proposed that the metric is afunction of a difference of waypoints and/or speeds and/or accelerationsat waypoints on the respective trajectories.

In particular, the difference can be calculated in terms of location,speed, and acceleration at three times.

Such a relationship may be described with the following formula:

${M\left( {t_{k},t} \right)} = {{\sum\limits_{T}^{{t\; 1},{t\; 2},{t\; 3}}\left( {{v\left( {t_{k},T} \right)} - {v\left( {t,T} \right)}} \right)^{2}} + \left( {{x\left( {t_{k},T} \right)} - {x\left( {t,T} \right)}} \right)^{2} + \left( {{a\left( {t_{k},T} \right)} - {a\left( {t,T} \right)}} \right)^{2}}$

where M(t_(k),t) represents the metric of trajectory t_(k) with:t_(k)=possible trajectories; t=observed trajectoriesv(t_(k),T) is the speed on the trajectory t_(k) at time T: (t1,t2,t3);x(t_(k),T) is the location on the trajectory t_(k) at time T:(t1,t2,t3);a(t_(k),T) is the acceleration on the trajectory t_(k) at time T:(t1,t2, t3).

The variable t_k represents the kth trajectory, t represents theobserved trajectory, t1, t2, t3 represent three time steps, v representsthe speed, x represents the location and a represents the acceleration.

According to one aspect, it is proposed that the recommendation serviceis a collaborative recommendation service.

According to one aspect, it is proposed that the adaptation or trainingof the recommendation system is done offline.

According to one aspect, it is proposed that the recommendation serviceshould assess the at least one possible trajectory using a neuralnetwork which is trained using a large number of assessed observedtrajectory-traffic environment combinations of road users.

This makes it possible for the trained neural network to directlyprovide a scalar as a measure for assessing possible trajectories at itsoutput.

A neural network provides a framework for many different algorithms formachine learning, for collaboration, and for processing complex datainputs. Such neural networks learn to perform tasks using examples,typically without being programmed with task-specific rules.

Such a neural network is based on a collection of connected units ornodes referred to as artificial neurons. Each connection can transmit asignal from one artificial neuron to another. An artificial neuron thatreceives a signal can process it and may then forward a modified signal.

In conventional implementations of neural networks, the signal on aconnection of artificial neurons is a real number, and the output of anartificial neuron is calculated by a nonlinear function of the sum ofits inputs. The connections of artificial neurons typically have aweighting that adjusts as learning progresses. The weighting increasesor decreases the strength of the signal on a connection. Artificialneurons can have a threshold, so that a signal is only emitted when theentire signal exceeds that threshold. Typically, a large number ofartificial neurons are combined in layers. Different layers may performdifferent types of transformations for their inputs. Signals migratefrom the first layer, the input layer, to the last layer, the outputlayer, possibly after passing through the layers multiple times.

The architecture of an artificial neural feed-forward network can be anarchitecture which is configured to receive a single data pattern at itsinput stage, for example corresponding to an image, and which providesan output value, for example a classification vector or a scalar, which,according to the problem, contains an approximation based on previoustraining. Feed-forward networks pass through each neuron exactly once.

A multi-layer perceptron (MLP) belongs to the feed-forward artificialneural network family. In principle, MLPs consist of at least 3 layersof neurons: an input layer, an intermediate layer (hidden layer) and anoutput layer. This means that all neurons of the network are dividedinto layers, wherein a neuron of a given layer always receives itsinputs only from a single previous layer and also forwards its outputsonly to a fixed layer. Accordingly, there are no connections which skipa layer, or which are activated multiple times. Apart from the inputlayer, the different layers consist of neurons which are generallysubjected to a nonlinear activation function and which are connected tothe neurons of the next layer.

A recurrent neural network (RNN) is a neural network which, unlike thefeed-forward networks, also has connections from neurons of one layer toneurons of the same or a previous layer. This structure is particularlysuitable for discovering time-coded information in the data.

In addition to the feed-forward neural network implementations above,there is the construction of an artificial neural convolutional networkfrom one or more convolutional layers, possibly followed by a poolinglayer. The sequence of layers can be used with or without normalizationlayers (for example batch normalization), zero-padding layers, dropoutlayers and activation functions, such as rectified linear unit ReLU,sigmoid function, tanh function or softmax function.

In principle, these units can be repeated as often as desired, whereinin the case of sufficient repetitions deep convolutional neural networksare then referred to. After some repetitive blocks consisting ofconvolutional and pooling layers, the CNN can be terminated with one (ormore) fully-connected layers, similar to the architecture of the MLP.

For the K-nearest neighbor method, vectors with the information from theenvironment capture, a map for autonomous driving and information abouta possible trajectory are stored in a vector space during the adaptationor training.

The vector space is spanned by the length of the input vector, forexample with 80 entries for environmental capture and 20 for trajectory,the vector space would span R{circumflex over ( )}100. R represents thereal numbers here and can be reduced depending on the dimension of theindividual entries, for example to the natural numbers for discretevalues or binary inputs.

For assessing a new environment capture-trajectory combination,represented by a vector, the new vector is compared with the existingvectors using a metric, for example the Euclidean norm. The existing“k”-nearest vectors are added to the assessment.

The rating is assessed as the weighted sum of the ratings of theK-nearest neighbors. For example, the same weightings or weightingsselected inverse to the distance can be used as weightings.

This means that the capture of the traffic environment, including staticinformation from map material, will be compressed to a vector by meansof an autoencoder. In addition, directly calculated measures of theenvironment, for example traffic light status, right-of-way rules, speedlimits, are stored in a separate vector. The autoencoder provides avector of size 256, the measure vector size is 64.

In addition, the trajectory is represented by means of a vectorcompressed by an autoencoder and a measure vector. The sizes of thevectors are 64 and 24, respectively.

The combined vector of the environment capture and the trajectoryconsists of 408 entries and spans a vector space of this size.

For each vector entry there is an assessment for the training time basedon the similarity of the trajectory actually driven and the possible,predicted trajectory.

At the inference time, for online assessment, the assessments of the 3nearest neighbors of the predicated trajectory are arithmeticallyaveraged for a given environment.

For runtime optimization, the spanned vector space can be compressed bymeans of a main component analysis and then partitioned by means of“trees” so that the next entries can be found more quickly.

For the recommendation service, features of the traffic environment withassociated observed trajectories are transformed into a feature spaceand stored. For example, by a K-nearest neighbor method, the mostprobable trajectories are determined on the basis of distances in thefeature space. This does not take the next class, but the class whosefeatures are closest to the current trajectory.

According to one aspect, it is proposed that the assessment of possibletrajectories is carried out repeatedly, for example cyclically. Thiswill enable reactions to changes in the traffic situation.

For example, new assessments are performed whenever one of theasynchronously determined traffic environments has been calculated.

A method for planning a trajectory to be driven by at least one of theroad users in a traffic environment is proposed, wherein at least onepossible trajectory for each road user in the traffic environment isdetermined in one step.

In a further step, at least one of a plurality of each possibletrajectory for each road user is assessed in accordance with the methoddescribed above. For example, a possible trajectory per reachable laneand a plurality of extra scenarios.

In a further step, the trajectory to be driven of the at least one roaduser in the traffic situation is determined using all the assessed,possible trajectories of the other road users.

According to one aspect, it is proposed that the assessments of possibletrajectories are transferred to a trajectory planning system. This thenallows the trajectory planning system to plan a trajectory to be driven.

According to one aspect, it is proposed that depending on the assessmentof the trajectory of at least one of the vehicles, its ego-trajectory iscalculated based on all the information that is available.

The described method can be used for behavioral planning, trajectoryplanning, motion planning, as well as an emergency braking assistant oras a driver assistance system.

From the assessments of the possible trajectories, probabilities can becalculated and planning of an ego-trajectory to be driven can then becarried out with the possible trajectories identified in this way.

A system for proposing an ego-trajectory to be driven is proposed, whichis set up to perform the method in accordance with the method describedabove and, depending on the assessment of the trajectories of the otherroad users, proposes a trajectory to be driven to one of the road users.The trajectory to be driven is the trajectory that the road user wantsto use to reach his destination.

An apparatus is proposed which is set up to perform the method describedabove.

A computer program is proposed that includes commands which, when theprogram is executed by a computer, cause it to perform the methoddescribed above.

A machine-readable storage medium is proposed on which the computerprogram described is stored.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the disclosure are shown in FIGS. 1 to 3 andare explained in more detail below. In the figures:

FIG. 1 shows an intersection with possible trajectories for a road user;

FIG. 2 shows a junction with two road users and possible trajectories;and

FIG. 3 shows the steps of the method for assessing possibletrajectories.

DETAILED DESCRIPTION

In FIG. 1, a traffic environment 100 is outlined in the form of a planview of a multi-lane intersection. A road user 110 is sketched, as wellas a variety of possible trajectories for the road user 110 marked withdashed lines a to f Multiple trajectories for at least one lane can alsobe taken into account. Previously noted road users are only indicated bythin lines of their observed trajectories.

FIG. 2 outlines another traffic environment 200 in a plan view of aT-junction or a junction with an ego-vehicle 210, which is not entitledto drive on, in front of a stop line 211. From the point of view of theego-vehicle 210, a road user 220 is coming from the left, whose vehicle2 is at a slight angle to the straight ahead direction on the priorityroad. In the situation of the road user 220, there are three options forcontinuing to drive on and a possible trajectory is marked for eachoption. For turning into the side road the trajectory is 230, forcontinuing straight ahead the trajectory is 250 and for turning round onthe priority road the trajectory is 240. All possible changes ofdirection are calculated based on the lanes present and possible merges.In addition, the trajectory to be driven of the ego-vehicle 210, whichis planning to turn left onto the priority road, is marked. The timest1, t2, t3 are indicated on the possible trajectories 230, 240, 250 ofthe road user 220 by black dots.

In addition, the observed trajectory 270 of the road user 220 is shownand the times t1, t2, t3 are indicated with small squares on theobserved trajectory 270 to describe steps of the adaptation of therecommendation service.

In accordance with the method for assessing possible trajectories ofroad users in a traffic environment, in one step the traffic environment200 is captured with the static features of the T-junction using ageographical map, for example a plan view, and the dynamic features ofthe ego-vehicle 210 and the road user 220. A series of features, forexample relative distances of the vehicles 210, 220: barely a vehicle'slength, the number of road users: here it is two, the spatial positionthereof relative to each other: here it is almost transverse to eachother, traffic rules: here a road with priority, and the stop line asthe inputs for the recommendation service for the assessment of allpossible trajectories 230, 240, 250.

In a further step, a road user 220 is identified by means of thecaptured dynamic features of the traffic situation 200, for example. Ina further step, the possible trajectories 230, 240, 250 are determinedfor the at least one road user 220 of the traffic situation 200. In afurther step, the possible trajectories 230, 240, 250 for the at leastone road user 220 of the traffic environment 200 are assessed by arecommendation service. During this, the possible trajectories 230, 240,250 were determined in particular while taking into account drivingcomfort on the trajectory and compliance with traffic rules, as can beseen from FIG. 2.

The recommendation service assesses the possible trajectories 230, 240,250 using a variety of assessed observed trajectory-traffic environmentcombinations of road users, as described above.

If the road user 220 has continued his journey, the observed trajectory270 marked in FIG. 2 may be determined to obtain furthertrajectory-traffic environment combinations for training therecommendation service.

The observed trajectory 270 can be assessed using a metric with thepossible trajectories 230, 240, 250 by determining the waypoints on therespective trajectories at three times and measuring the distancebetween these waypoints of the observed trajectory 270 and therespective possible trajectory 230, 240, 250. The sum of these valuesgives a measure of the assessment according to formula 1.

The recommendation service used has an adapted, in particular trainedneural network, wherein the recommendation service assesses the at leastone possible trajectory using a neural network trained with the largenumber of assessed observed trajectory-traffic environment combinationsof road users. The neural network has a first autoencoder for capturingthe environment and a second autoencoder for capturing the possibletrajectories.

The outputs of the environment capture are used as inputs for anautoencoder which is used to represent the current traffic situationaround the ego-vehicle and the road users.

The outputs of the environment capture are partly transformed into aplan view by means of 3D transformations and accordingly represented in2D as image planes. These image planes are then compressed byconvolution planes as part of an autoencoder.

In addition, information such as traffic rules, distances between roadusers, speeds, traffic light states, and other difficult-to-visualizeproperties are added as vectors to a rear layer of the autoencoder.

The possible trajectories are also handled using 3D projections as 2Dvisualizations including map material in a separate autoencoder andsupplemented by an additional information vector for the possibletrajectory.

The outputs of the two autoencoders are then combined and routed into asystem of multiple layers to dense layers and normalization layers. Theoutput of this system is the scalar assessment of the possibletrajectory for the current environment. Additional dropout layers switchoff some neurons in the neural network randomly to reduce thepossibility of overmatching.

A visualization of different properties of the environment capture ispassed as an input into an autoencoder with a specified base structure.The environment is defined in a (6, 224, 224) tensor, which is definedas visualization of the environment as a 6 color channel image with astructure of resolution 224 pixels×224 pixels. The different colorchannels represent static structures, map information, dynamicstructures, speeds, current right of way lanes, and traffic lightphases.

The base structure of the convolution network is defined as RESNET-34with an output vector of 2000 units. In addition, we add 96 units withadditional information about the environment, such as speed limits,right of way rules, accurate distances, and lane association.

The trajectory is visualized as a two channel image. One channelvisualizes map material and the other a possible trajectory. Anautoencoder using Resnet-34 encodes the visualization as a 512 unitvector. Additionally, one-dimensional speed, acceleration, andorientation steps are added in a 72-unit vector.

The outputs of the autoencoders with information vectors, 2096 and 584units, are output as a vector into a dense feed-forward network with ascalar output. The structure of this network is as follows.

TABLE 1 Layer (type) Output form Parameter # Input (2680, 1) 0 Dense 21(4096, 1) 4096*2680 Dropout (4096, 1) 0 Dense 2 (2048, 1) 2048*4096Batch Normalization (2048, 1) 0 Dense 3 (1028, 1) 1028*2048 Dense 4 (512, 1)  512*1028 Output (1) 512 

Table 1 describes the feed-forward network behind the autoencoders. Theinput is a 2680 unit input vector.

What is claimed is:
 1. A method for assessing possible trajectories ofroad users in a traffic environment comprising: capturing the trafficenvironment with static and dynamic features; identifying at least oneroad user; determining at least one possible trajectory for the at leastone road user in the traffic environment; and assessing the at least onedetermined possible trajectory for the at least one road user using anadapted/trained recommendation service and the captured trafficenvironment.
 2. The method according to claim 1, wherein therecommendation service is adapted to assess possible trajectories usinga large number of corresponding combinations of assessed observedtrajectories, different traffic environments, and the at least onedetermined possible trajectory.
 3. The method according to claim 1,wherein the recommendation service is a collaborative recommendationservice and has a neural network with at least one convolution layer ora recursive neural network or is based on a K-nearest neighbor method.4. The method according to claim 2, wherein: the recommendation serviceis a collaborative recommendation service and has a neural network, andthe neural network has a first autoencoder configured to capture thetraffic environment and a second autoencoder configured to capture thepossible trajectories.
 5. The method according to claim 2, wherein: therecommendation service is a collaborative recommendation service and isbased on a K-nearest neighbor method, and vectors of the K-nearestneighbor method are formed according to a large number of correspondingcombinations of the captured traffic environment and the observedtrajectories.
 6. The method according to claim 1, further comprising:determining the at least one possible trajectory using a geographicalmap of the traffic environment.
 7. The method according to claim 1,further comprising: capturing the traffic environment by transformingspatial parts of the traffic environment into a two-dimensionalreference system, corresponding to a plan view of the trafficenvironment.
 8. The method according to claim 1, wherein the capturedtraffic environment includes a previous trajectory of at least one roaduser.
 9. The method according to claim 1, further comprising:determining the at least one possible trajectory according to anoptimization of cost functions, a search-based method, and/or amachine-learning method.
 10. A method for generating a recommendationservice for assessing possible trajectories, comprising: determining alarge number of corresponding combinations of captured trafficenvironment, observed trajectory, and at least one possible trajectoryfor a large number of different traffic environments, by repeatedly:capturing the traffic environment with at least one static feature andat least one dynamic feature; identifying at least one road user in atraffic situation; capturing an observed trajectory of the at least oneroad user; determining at least one possible trajectory for the at leastone road user in the traffic environment; and adapting therecommendation service with the large number of correspondingcombinations of the captured traffic environment, the observedtrajectory, and the at least one possible trajectory, using a deviationof the observed trajectory from the at least one possible trajectory.11. The method according to claim 10, wherein for an assessment of anobserved trajectory-traffic environment combination in a capturedtraffic environment, a difference of a possible trajectory of the atleast one road user from the observed trajectory is calculated by usinga metric.
 12. A method for planning a trajectory to be driven by atleast one road user of a traffic environment, comprising: determining atleast one possible trajectory for each road user in the trafficenvironment; assessing a plurality of trajectories for each road userby: capturing the traffic environment with static and dynamic features,identifying at least one of the road users, determining at least onepossible trajectory for the at least one road user in the trafficenvironment, and assessing the at least one determined possibletrajectory for the at least one road user using an adapted/trainedrecommendation service and the captured traffic environment, determiningthe trajectory to be driven by the at least one road user of the trafficenvironment using all the assessed, possible trajectories of other roadusers.
 13. The method according to claim 12, wherein an apparatus is setup to perform the method.
 14. The method according to claim 12, whereina computer program, comprising commands which, when a computer executesthe program, causes the computer to perform the method.
 15. The methodaccording to claim 14, wherein the computer program is stored on amachine-readable storage medium.